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ABSTRACT 

In order for learning providers to become learning 
organizations, their services must be delivered effectively. The Learning 
PROBE (Promoting Business Excellence) was created for the learning and skills 
sector and was in use by nearly 100 colleges and work-based learning (WBL) 
providers by April 2003. PROBE has assisted 3000 organizations across the 
world in reviewing their performance and practices by measuring themselves 
against their peers, against world-class benchmarks of service excellence, 
and against the achievements and practices of organizations in other sectors. 
Key findings based upon the first 48 learning providers to complete the 
Learning PROBE benchmarking process are as follows : ( 1 ) the application of 

good practice leads to improved service and business performance; (2) none 
have yet achieved the standards of a "world-class'* service organization, but 
46% are contenders for this status; (3) good performance that is not 
underpinned by good practice has resulted in outcomes achieved at a high cost 
in financial and human terms; (4) strongest practices relate to the staff and 
their contribution to success; (5) colleges are results-oriented, with 
strengths apparent in business performance, productivity, and service 
quality; (6) similar patterns of strength and opportunity will likely 
characterize WBL providers; and (7) one of the weakest practice areas is 
continuous improvement. (Contains 6 figures, 10 tables, 8 references, and an 

appendix entitled "Taking Part in Learning PROBE.") (MO) 
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Foreword 



As a college governor over the last 15 years I have been part of 
a process that has seen a succession of business people like me 
attempting to bring their commercial experiences to bear on the 
world of further education. It has been a journey in which we have 
all been learning. Not all of our worthy attempts to use familiar 
business approaches have found a natural home in the management 
of a large education facility. What we do have in common is 
a passion for meeting and exceeding the expectations of our 
customers. However impressive our post-incorporation building 
programmes, it is clear that our real achievements will never be 
better than the quality of the student experience. We are in 
a competitive world in which we need to embrace anything we 
can in order to help us systematically make the Journey towards 
organisational excellence. 

The PROBE tool was first developed in 1992 to help manufacturing 
companies to emulate the Japanese shop floor miracle of the 
1980s. My team in IBM gave birth to the approach with the essential 
help and inspiration of Professor Chris Voss at the London Business 
School. Since then thousands of manufacturing sites across 
the world have made great strides towards becoming world class 
and I have had the privilege of seeing incredible transformations that 
started with an honest self-assessment using PROBE. The enemy of 
progress has often not been ignorance but complacency. We typically 
think we are much better than we really are. PROBE seeks to 
separate the rhetoric from the reality. Since its original inception 
PROBE has been successfully developed for the services, health, 
education and agriculture sectors. 

In the early days of developing Learning PROBE it was piloted at 
my own college and it was with enormous pride that some years later 
I was able to see the college principal collect the Midlands 
Excellence Award at a glittering ceremony at the Birmingham 
International Convention Centre. There must, however, be no 
complacency. This is one of those Journeys in which the further 
you go, the more you see there is still to do. It is a journey that 
has no end. 



All colleges must aspire to being ‘excellent colleges’ and we are 
indebted to Jane Owen and her colleagues at the Learning and Skills 
Development Agency and to David Yarrow and his team at the Centre 
for Business Excellence at the Northumbria University for making 
this approach accessible to us all. This report begins the process 
of sharing the findings from the programme as a whole. I hope that 
those colleges and other learning providers who have not yet used 
Learning PROBE will be encouraged to do so and that those who have 
will find continued energy for their journey. 



Philip Hanson 

Chairman of Governors, Solihull College 




The publication 

This publication is intended to explain how PROBE was developed 
and adapted for the learning and skills sector, resulting in the use 
of Learning PROBE and its small business version by nearly 100 
colleges and work-based learning (WBL) providers by April 2003. 

It also reflects the lessons learnt by the first 48 learning providers to 
complete the Learning PROBE benchmarking process. Though the 
initial group are all colleges, the programme has now been adapted 
and expanded to include WBL providers, and the early results are 
also reported. At the time of the analysis, many more WBL providers 
and colleges were preparing to participate, and local Learning and 
Skills Councils (LSCs) were working with the Learning and Skills 
Development Agency (LSDA) to ensure that they and their local 
providers benefited from the opportunity to review, to learn, to plan 
and to improve - in other words, to make continuous improvement 
a reality throughout the learning and skills sector. 



Learning has never been as important as it is in today’s turbulent 
world. It is fundamental to everything we do, in every walk of life, 
at every stage of life. Learning holds the key to many of the problems 
and challenges which beset us as individuals, as a society, and as an 
economy. In many ways, our future depends upon the delivery 
of that most important of visions - a community of lifelong learners, 
a learning society. 
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To fulfil the role of a learning provider is both a privilege and 
a responsibility. So much depends on the provider's ability to 
inspire, to deliver, to support, to serve the needs of its learners ... 
its customers. Learning providers are organisations managed and 
staffed by dedicated people who care deeply about their learners, 
and do everything in their power to facilitate effective learning. 

They strive to improve their skills, their knowledge, their resources, 
their approaches, their delivery and their support processes. 

If learning providers can themselves become learning organisations, 
their services will be delivered more and more effectively, and 
one of society's most important functions will be one of the best 
fulfilled. If LSCs, and others who commission and quality assure 
the work of learning providers, can also effectively support their 
further development, then progress towards that goal will be 
considerably enhanced. 

With this in mind, the LSDA set out to help learning providers 
to benefit from diagnostic tools and knowledge bases, assisting 
providers to analyse their service delivery and the quality of 
their internal processes. The LSDA selected PROBE, a suite 
of benchmarking tools, and worked with its creators to develop 
Learning PROBE, a version that is user-friendly and helpful for 
organisations and staff in the learning and skills sector. 

PROBE is a tool that has so far assisted 3000 organisations 
across the world to review their performance and practices. 

The identity of each participant, and their individual data, are 
strictly confidential; however, each new participant can compare 
their practices and performance against an extensive and rich 
database, representing experiences and ideas drawn from 
organisations in many sectors which are committed to learning 
and improving. Learning providers can measure themselves against 
their peers, against world-class benchmarks of service excellence, 
and against the achievements and practices of organisations in 
other sectors, whose different approaches to generic issues can 
stimulate fresh insights and ideas. 
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1 The experiences of the first 48 colleges confirm that the 
application of good practice leads to improved service and 
business performance. 

2 None of the colleges has yet achieved the standards of a 
‘world-class’ service organisation; 46% are contenders for 
world-class status, which is encouraging, but suggests that 
the sector has some work to do to catch up with other parts 
of the UK service sector. 

3 A high proportion display good performance that does not appear 
to be underpinned by good practice, suggesting that outcomes 

are achieved at a high cost in financial and human terms, leaving the 
organisation somewhat vulnerable and ill-prepared for further 
development or growth; findings from a separate staff survey, 
involving some of the same colleges, support this interpretation. 

4 Colleges’ strongest practices relate to the staff and their 
contribution to success; empowerment and innovativeness are 
strengths, but the virtuous cycle of people management practices 
which characterises leading service organisations is less evident. 

5 Colleges are results-oriented, with strengths apparent in business 
performance, productivity and service quality; however, some 
colleges are reluctant to commit to clear, challenging service 
standards, and in some cases internal perceptions of strong 
performance are open to question given the lack of evidence 
gathered through performance measurement. 
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6 The biggest opportunity for improvement is in the service processes 
which comprise the engine room of the colleges’ service delivery; 
few are advanced in their management of processes, with fewer 
still delighting customers at those crucial ‘moments of truth’. 

7 Early indications are that similar patterns of strength and opportunity 
will characterise WBL providers; staff empowerment 

and involvement are emerging as strengths, while process 
management and organisational performance management 
exhibit room for improvement. 

8 The challenge for colleges, and for the sector as a whole, is 
encapsulated in the fact that one of their weakest practice areas 
is continuous improvement; if the sector is to close the practice 
and performance gap on other service providers, it must first catch 
up with modern practices in the field of quality, improvement 

and the management of change. 



Summary of colleges’ strengths 
and opportunities for improvement 

Table 1 presents details of strengths and opportunities for 
improvement identified within the colleges as a group, through 
the five key scales used to interpret the findings derived through 
Learning PROBE. 
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Learning excellence 



TaW© 1 Colleges’ strengths and opportunities for improvement 



Area 


Strengths 


Opportunities for improvement 


Leadership 


Relationship building 


Management style 




Customer relations 


Problem solving 




Staff attitude 


Influencing suppliers of workforce 




Management involvement 
in quality leadership 


Value for money 
Elimination of ‘waste’ 




Quality values 


Benchmarks 




Supplier relationship and stability 
Outsourcing 


Measurement of staff satisfaction 
Management reward 


Service 

processes 




Accessibility 

Organisation structure for service 
and product development 

Customer Input for products and services 

Management of key processes 

IT as basis for knowledge management 

IT Integration 

Electronic commerce 

Problem-handling strategy for service recovery 
Education and training for quality 
Service guarantees and warranties 
Attention to staff loyalty 
Managing ‘moments of truth’ 


Peopie 


‘Real-time’ staff handling of problems 


Recognition and reward 




Learning organisation 


Employee involvement 




Strategic role of innovation 




Performance 


Management of customer loyalty 


Performance measurement and reporting 


management 


Management attention to quality measures 


Understanding markets 




Clarity of goals 


Visibility and communication of standards 




Customer satisfaction measurement 


Support for staff 




Challenging standards 


Established service standards 


Results 


Cash flow 


Clarity of service concept 



Return on assets 


Staff satisfaction 


Overall productivity within organisation 
Level of customer satisfaction 
Courtesy 
Reliability 

Value (quality/price) 

Trends in customer satisfaction 
Customer base 
Customer retention 


Impact on society 
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Rigomus 

seUfsissessment 



be an Integml pairt 



LSC (2001a) 
Self-assessment and 
development plans. 



The post-16 learning sector has experienced considerable change 
in recent times. College managers have had to develop new skills 
while taking on many new responsibilities since incorporation. 
There has been a sizeable increase in the number of people taking 
part in further education and service expectations have grown. 

The sector has expanded to include WBL providers as well as 
colleges, funded through the 
LSC. Throughout this period 
of change the government 
has continued to focus 
on encouraging quality 
improvement in the sector. 

The ongoing emphasis 
on meeting customer and 
stakeholder needs, together 
with increases in expectations, 
means that learning providers 
have to continuously improve 
their levels of service provision. 

The Further Education Funding Council introduced self-assessment 
as a requirement into the FE sector in England in 1994, and it 
is now very much a part of colleges’ management, enforced by 
the requirement to provide self-assessment reports (SARs) for 
the LSC and for inspection purposes, which are subsequently used 
during college inspections. Colleges also have to report on and 
set targets for key output performance indicators such as national 
benchmarks pertaining to student retention and achievement. 

The LSC has continued this policy within the newly enlarged sector 
and has produced some supporting publications. 



Self-assessment shouM be 
a systematic pmcess In which 
pmvMe^s collect anti analyse 
evidence In OFtieF to make 
lutigements about thelF 
peFfoFmance In Felatlon 
to agFeeti goals. The main 
puFpose of self-assessment 
Is self-ImpFovement. 

LSC (2001b) Guidance on self-assessment 
and development planning. 
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What is benchmarking? 

Metric benchmarking comparisons of outputs which can 
identify stronger and weaker aspects of performance. 

Diagnostic benchmarking helps to pinpoint processes/ 
practices which are causing stronger/weaker performance. 

Process benchmarking an advanced approach - 
practices which work well elsewhere are 
studied for potential adoption/adaptation. 

See Jane Owen’s (2002) guide 
Benchmarking for the 
learning and skills sector. 



Self-assessment reporting and target setting are useful methods 
of measuring quality, and stimulating thinking about achievement 
and improvement. However, while the results facilitate comparisons, 
these methods do not detail processes in a way that allows colleges 
to readily identify and compare good practice in a systematic 
and measurable way. By 1999, when the Raising Quality and 
Achievement (RQA) Programme was set up as part of the LSDA, 
the sector was used to working with metric benchmarks, but had not 
used diagnostic or process benchmarking to any significant degree. 
To further enhance the use of benchmarking within the sector it was 
decided to either develop a learning sector diagnostic benchmarking 
tool or to adapt one that was already successful in other sectors. 
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Learning PROBE is one of the iatest deveiopments in a suite 
of benchmarking toois deveioped over the course of a decade. 

The PROBE toois are recognised woridwide as a ieading approach 
to business assessment, deployed through a network of highly 
credible agencies, which facilitate assessments with organisations 
of many types in many countries and sectors, buiiding up 
a formidabie set of databases and iearning opportunities. 

PROBE began life with the ‘Made in Europe’ study (Hanson et al. 
1994), an initiative of London Business Schooi (LBS) and iBM 
Consuiting in the eariy 1990s. This study provided insights into 
the heaith of UK manufacturers and their European counterparts, 
which made a major impact on the UK’s national industrial agenda, 
as evidenced by its contribution to the government’s 
Competitiveness White Papers. 

Through the mid- to iate-1990s, the PROBE suite deveioped, 
to incorporate versions tailored to most sizes and types of 
organisation, across the fuii range of sectors. 
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Development of the PROBE suite since the mid-1990s 

□ After ‘Made in Europe’, Manufacturing PROBE rolls out 
internationally, enhancing database size and quality and 
extending reach to every continent and industrial sector. 

B LBS works with the universities of Southern California and 
North Carolina to adapt methodology for a major UK versus US 
comparison of the service sectors, spawning ‘Service PROBE’. 

3 CBI adopts the tools and promotes them nationally as 

the PROBE benchmarking service - PROmoting Business 
Excellence. Results feed policy formulation, lobbying and 
good practice exchange and dissemination activities. 

a Newcastle Business School Joins the PROBE Partnership, 
deploying an adapted version to achieve mass participation 
in benchmarking by companies in north-east England. 

a The Partnership, with contributions from West London TEC 
and Business Link County Durham, develops ‘Microscope’ 
and ‘Micro Business Review’ tools, tailored to needs of small 
to medium and micro-sized organisations. 



In recent years PROBE has been adapted to a number of sectors 
including health, agriculture, food and the learning and skills sector. 

Countries represented in the PROBE databases 



Australia 


Hungary 


Poland 


Belgium 


Iceland 


Portugal 


Brazil 


India 


Russia 


Canada 


Indonesia 


South Africa 


China 


Ireland 


Spain 


Cyprus 


Italy 


Sweden 


Denmark 


Malaysia 


Switzerland 


Estonia 


Mexico 


Taiwan 


Finland 


Netherlands 


UK 


France 


Norway 


USA 


Germany 


Philippines 
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The PROBE databases contain 10 years’ worth of research data. 

The data has drawn upon some 3000 individual site benchmarks 
from 32 countries, with new data being added constantly. 

The PROBE benchmarking process has been refined and improved, 
ensuring that new participants derive invaluable comparisons 
and insights, and identify clear improvement opportunities. 

PROBE is built around a series of insights into the true meaning of 
organisational excellence, examining the processes and practices an 
organisation deploys and the performance results it achieves. The 
model is based upon extensive research into the key features 
of organisations which deliver world-class service. 

Excellent leadership creates an environment in which the 
organisation’s people contribute to their full potential, deploying 
effective and efficient service processes and managing 
performance in order to ensure the achievement of optimal 
results. The PROBE model promotes the Fundamental Concepts 
of Excellence, themselves the basis of the EFQM Excellence 
Model (2003), and in many organisations is used hand-in-glove 
with that model as complementary tools to assist organisational 
assessment, learning and improvement. 

The Fundamental Concepts of Excellence 

■ Results orientation 

■ Customer focus 

■ Leadership and constancy of purpose 

■ Management by processes and facts 

■ People development and involvement 

■ Continuous learning, innovation and improvement 

■ Partnership development 

■ Public responsibility © EFQM 2003 European Foundation 

for Quality Management 
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Service PROBE began life with an Anglo-American International 
Service Study , conducted in the mid-1990s by Professor Chris Voss 
of the London Business School with colleagues from business 
schools on the east and west coasts of the USA (Voss et al. 1997). 
The study assessed the management practices of individual 
service organisations and the resulting indicators of performance 
outcomes, drawing upon and in turn building upon the findings 
of the ‘Service-Profit Chain’ research (Heskett et al. 1994, 
see Figure 1), studies which changed the way service excellence 
is viewed and achieved. 



Fogur© 1 The service-profit chain 




The research highlighted the importance of customer retention, 
finding that in some sectors a 5% increase in customer retention can 
lead to as much as 100% improvement in business performance. It 
also found that the key reasons for customer disloyalty are 
associated with poor service recovery when things 
go wrong, discontent setting in if the customer has to deal with 
more than one person to get action and if the resolution takes an 
excessive amount of time. By contrast the intention to repurchase 
by a customer who has experienced excellent service recovery is 
almost as high as if nothing had gone wrong. The study also found 
that organisations which enjoy high customer satisfaction and loyalty 
also enjoy high employee satisfaction and loyalty. 
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Leammg P^O&E 
pmmded us wit Hu 
a summary off 
our stresngtlnis 
and w®akn®ss®s 
against a wid®r 
ffram®work than 
thos® w® had 
pr®viousiy us®d 
and ®naM®d us to 
ass®ss all ar®as off 
th® €oll®g® against 
on® fframowork. 

FE college, 2003 
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Learning PROBE - adapting for the sector 

The development of a learning and skills version of Service PROBE 
was carried out jointly by the RQA Programme at the LSDA and 
Newcastle Business School at Northumbria University. Eighteen 
colleges contributed to a consultation exercise, through which the 
benchmarking tool was adapted to the specific needs and language 
of the sector. The Learning PROBE programme began during the early 
months of 2000, and there has been a steady stream of colleges 
participating ever since. 

During 2002, in order to ensure that WBL providers could also easily 
access the service, a further pilot was undertaken, with the support 
of Tyne and Wear LSC. As with the original pilot, the providers that 
took part all volunteered. They ranged from small local organisations 
to regional offices of national organisations, and provided feedback 
which drove a review of the Learning PROBE tool and the introduction 
into the sector of a tool tailored for organisations employing 
20 people or fewer -the Micro Business Review (MBR). 



Learning PROBE and MBR both depend upon the involvement of 
a suitably sized team, representing a ‘diagonal slice’ through the 
organisation. The team works with an experienced, independent 
facilitator to develop a consensus view of how the organisation 
measures up to a range of templates of good practice and strong 
performance. The facilitator’s input ensures that a true consensus 
is developed, based on the realities of day-to-day life in the 
organisation, and that it represents a realistic view, truly comparable 
to those of other organisations in the PROBE databases. 



The facilitator provides feedback to the team and their 
organisation through a presentation and a written report, and 
proceeds to help with the process of interpreting the emerging 
lessons and planning appropriate improvement actions. A strength of 
the PROBE approach to benchmarking is that it works well on 
two levels - helping individual learning providers to reflect, learn 
and improve, while simultaneously conducting an invaluable study 
of the sector as a whole - its strengths, opportunities and 
development needs. 
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The Learning PROBE project 



Learning excellence, the foundation of the PROBE concept, is 
based on the proposition that good practice in the management of 
processes and services leads to high levels of service performance, 
which in turn leads to superior performance outcomes. 

Learning PROBE assesses the practices of individual learning 
providers and the resulting indicators of performance outcomes, 
as perceived and measured by the managers and staff of those 
organisations. 

□ Practice refers to the established processes a provider has in place 
to design, deliver and measure its service. These include service 
design such as the design of new courses, service recovery, for 
example dealing with a learner complaint and rectifying problems, 
staff involvement and discretion and service culture. 

□ Performance refers to the way a provider’s service quality can be 
measured, including its impact on overall performance outcomes. 
Examples of external service measures include employers' 
perceptions of the provider’s reliability and accessibility, and learner 
satisfaction, while internal measures include staff loyalty and 
morale. 

The Learning PROBE programme is ongoing, and by the autumn of 
2002 a total of 48 colleges had participated. These form the basis 
of the analysis presented here. They cover a broad range of college 
types, sizes and locations within England. Together, they represent 
more than 10% of all of the colleges in England. 
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Learning PRQBE was Feally 
useful In assessing ouf posstl&n 
m the key pmeesses, quality 
systems and pmcedums that 
we use In our ©allege. As a 
consequence, we have chosen to 
sustain om path to excellence by 
going down the EFQM Excellence 
Model mute. We have adopted 
the matrix appmach In this and 
the output from PROBE had a 
consMemMe Influence on this 
course of action, 

FE college, 2003 

their practices and performance. This potentially provides a set 
of benchmarking results that imply a better picture of the college 
sector as a whole, although the relative strengths and shortcomings 
indicated from the study may be representative of the wider picture. 

It is worth noting, however, that the same is true of other similar 
benchmarking schemes and studies, against which the colleges’ 
data and findings can be compared. 

Learning PROBE poses 91 questions to the learning provider, 
covering each of the areas of service practice and performance. 

Each answer is scored using a continuum scale from ‘1’ to ‘5’, with 
‘5’ representing world-class levels of implementation or attainment. 
To simplify discussion of the results an average score above ‘3’ per 
attribute is assumed to represent a ‘good’ level of implementation or 
outcome (equivalently, an index above 60 for a group of measures), 
while ‘very good’ levels are represented by scores of ‘4’ and above 
or indices of 80 -f. 



One key issue in interpreting 
the research data centres on 
the extent to which the sample 
of 48 participating colleges is 
representative of colleges in 
England as a whole. This sample 
was typically self-selecting, and 
as a consequence it can be 
argued that these colleges are 
likely to be more quality focused 
than perhaps a ‘typical’ sample 
of colleges. The participants 
also tend to be colleges that 
respond positively to offers 
of support in order to improve 
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The model on which Learning PROBE is based is shown in Figure 2. 
Major areas shown in the model are: leadership, organisation 
and culture, service concept and design, delivery and value. 

Q Leadership drives the whole service delivery process. Good 
leadership should emphasise the importance of both customers 
(including learners, employers and parents) and staff. 

0 Even a well-designed service will not meet the needs of customers 
unless it is delivered effectively: therefore, service delivery must 
include appropriate processes for managing the service and 
responding to customer complaints and problems. 

a Leadership contributes to a customer and service-oriented 
organisation and culture, which will, in turn, lead to greater staff 
loyalty and morale. 

□ Service concept and design is central to good service performance. 
Service concepts such as those for courses, learner support and 
learner assessment must be designed to meet customer needs 
and provide differentiation and distinction in the provider’s 
service offerings. 

B Service value, as perceived by customers, is created by a 
well-designed service concept and delivery system, combined 
with a satisfied, loyal and productive staff. 
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Fogyp© 2 Service practice and performance model 



Leadership 
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The service operation 

Organisation Service concept Service Service 

and culture and design delivery value 
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Outcomes 



Leadership 


Understanding 
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Managing costs 
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and value 


Staff 


Service 


Service 
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design 


recovery 


standards 




Service 
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process 


empowerment 
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Staff loyalty ^ Service 

and morale effectiveness 



Service 

quality 



Customer 

satisfaction 



Customer loyalty 
and retention 







Business performance 



The performance outcomes of each of these areas will be reflected 
in the organisation’s overall performance. Increasing customer 
satisfaction, loyalty and retention will increase long-term financial 
viability and, in turn, lead to long-term growth and long-term success 
in fulfilling the role of an outstanding learning provider. 
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Additional benefits 
of Learning PROBE 












pmcesses 



As well as the findings described, the study has shown that the 
participating colleges have been enthusiastic about using Learning 
PROBE. Apart from the obvious benefit of identifying individual 
strengths and opportunities for improvement, there have been a 
number of additional benefits. Learning PROBE encourages colleges 
to use staff feedback, following the example of the team approach 
used by PROBE. Many participating colleges are taking part in 
an organisational review for the first time and find it both a useful 
way to learn more about the college and an empowering experience. 
Participating employees tend to feel more comfortable 
with cross-college work and 
many have gone on to form a 
cross-college quality resource. 

One college used the members 
of the Learning PROBE team to 
develop target setting within 
the college: another used them 
to form the basis of a quality 
team. The systematic approach 
to reviewing processes has also 
benefited subsequent general 
quality reviews within colleges 
and approaches to inspection. 

A number of colleges have 
used the experience gained 
when dealing with external 
inspections. 



team has discuiBBed Its fmdmgB 
' has mcogmsed Sts vaSm m 



PROBE was a fecuB of a 



m OMF view, 
' BeotoF weak 
MBsng PROBE, 
fm BeSf-aBBeBBmeint 



FE college, 2003 
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Overview of results 

What do the Learning PROBE results to date tell us about practice 
and performance in English colleges? 

The analysis has examined the relationship between overall 
management practices in participating colleges and their overall 
performance. The data shows that there appears to be a strong 
relationship between practice and performance, supporting the 
hypothesis that good service management results in better service 
and business performance, and confirming the findings of earlier 
PROBE studies. Figure 3 indicates the strong association between 
the practice and performance indices in the sector.^ 



^ Statistically, the association is strong, as indicated by the measures 
(r = 0.825, p = 0.000). This suggests that the higher the practice index, 
the higher the corresponding performance index across the college sector 
as represented by the sample of participants. The significance value 
(p = 0.000 < 0.01) suggests we can be 99% confident that an association 
exists between the practice and performance indices. 
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Performance index 



Figure 3 Overall levels of practice and performance 




Practice index 



We can compare the spread of practices and performance in 
the sample of 48 colleges with the 155 UK service organisations 
benchmarked during the International Service Study (ISS). 
‘World-class’ service organisations are defined as those which 
have both leading management practices and performance equal 
to the world’s best. These organisations achieve a score of 80% or 
better in both aggregate practice and performance. Organisations 
below this score but with scores of over 60% for both practice and 
performance are considered to be ‘contenders’. 
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Fogmire 4 UK service organisations versus English colleges positioning 
on world-class practices and performance 

UK service 
organisations (%) 
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None of the colleges has yet achieved the standards of a ‘world-class’ 
service organisation (see Figure 4), though one college was very 
close, compared with 5% of UK service organisations which 
participated in the ISS. The sample of colleges contains a smaller 
proportion of ‘contenders’ for world-class status, and conversely 
a larger proportion whose overall practices and performance 
are ‘weak’. 

It is also noticeable that a relatively high proportion of the colleges 
display good performance that does not appear to be underpinned by 
good practice, suggesting that outcomes are achieved at a high cost 
in financial and human terms, leaving these colleges somewhat 
vulnerable and ill-prepared for further development or growth. 
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Practice and performance 

The colleges’ data has been examined in more detail by developing 
a number of scales and indices which provide insights into key 
aspects of practice and performance. The analysis pinpoints 
a number of practice and performance 
areas in which there are clearly 
opportunities for improvement. The 
analysis also identifies that a number 
of colleges demonstrate strengths 
in these same areas, and that overall 
there are some significant strengths 
upon which individual colleges and 
the sector as a whole can build. 

Five scales have been used to group the 
and performance for analysis purposes. 

g leadership 

■ service processes 
s people 

a performance management 

■ results. 

Overall mean scores have been calculated for each scale, and 
for subsets of the scales themselves. 

Table 2 suggests that the strongest practices relate to people, 
with the mean index across the sector being clearly in excess of 60. 
In contrast, the area perhaps requiring the greatest attention is 
service processes, with a mean index of only 56.3 and more than 
half of the sample scoring below 60. 



Learning PMBE gave 
the college management 
team an opportunity to 
evaluate and Improve 
our business and 
customer processes. 

FE college, 2003 

colleges’ practices 
These are; 
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Tatole 2 Overall profiles by practice and performance 



Overall score 


Leadership % 


Service 
processes % 


People % 


Performance 
management % 


Results % 


<50.0 


10 


31 


4 


15 


4 


50.0-59.9 


35 


29 


38 


25 


27 


60.0-69.9 


38 


33 


35 


44 


48 


70.0-79.9 


13 


6 


19 


13 


19 


80.0-89.9 


4 


0 


4 


4 


2 


90.0-100.0 


0 


0 


0 


0 


0 


Total 


100 


100 


100 


100 


100 


Mean 


61.2 


56.3 


63.0 


61.4 


63.6 



Table 2 also suggests that the colleges are results driven, with 
a clear majority experiencing good levels of overall performance. 

The belief that good practices potentially lead to high performance is 
supported by the results shown in Table 3. Associations between the 
results index and each of the four practice indices are strong and 
statistically significant, a point that is illustrated by Figure 5. 

There are also significant levels of association between all 
the areas of practice, which suggests that levels of practice 
implementation are consistent (either consistently weak 
or consistently strong) across the areas examined by 
Learning PROBE. 



Table 3 Association between groups of measures 





Service 

processes 


People 


Performance 

management 


Results 


Leadership 


0.884 ♦♦♦ 


0.837 ♦♦♦ 


0.856 ♦♦♦ 


0.809 ♦♦♦ 


Service processes 




0.705 ♦♦♦ 


0.817 ♦♦♦ 


0.777^^^ 


People 






0.744 ♦♦♦ 


0.743^^^ 



Performance management 0.749 ♦♦♦ 



♦ ♦♦ represents significance at the 0.1% level, ♦♦ at the 1% level and ♦ at the 5% level. 
All other correlations are statistically insignificant. 
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Leadership 



There are a number of areas that relate to college leadership 
(see Table 4). 

Relationship marketing is strong, with only 4% scoring poorly 
for relationship building or customer relations, and 96% of colleges 
having good levels of implementation, demonstrated by good 
relationships with many customers and the development of individual 
customer solutions. 

Lower levels of implementation are apparent for quality leadership 
and supplier relationships, with 33% and 34% of colleges showing 
poor attainment. Weaker practices include short-term supplier 
relationships, a passive role in influencing suppliers and outsourcing 
strategies having the effect of ‘hollowing’ the college. 



Table 4 Distribution of measures for leadership 



Overall 

score 


Quality 
leadership % 


Value 
orientation % 


Market 
acuity % 


Relationship 
marketing % 


Supplier 
relationships % 


<50.0 


2 


19 


19 


2 


17 


50.0-59.9 


31 


25 


31 


2 


17 


60.0-69.9 


46 


46 


31 


42 


44 


70.0-79.9 


17 


10 


15 


19 


8 


80.0-89.9 


2 


0 


4 


33 


15 


90.0-100.0 


2 


0 


0 


2 


0 


Total 


100 


100 


100 


100 


100 


Mean 


62.5 


56.9 


60.2 


68.5 


61.7 



While almost half the sample understand ‘value’ in service, maintain 
proactive support functions and have managers who are starting 
to ‘attack’ non-value-added activities, only one college has high value 
orientation. A total of 44% of colleges have poor levels of 
value orientation implementation, failing to focus their services 
and processes on what really matters to the customers. 
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Market acuity is the weakest aspect of 
leadership across the sector, with half 
of the colleges scoring poorly. Only 9% of 
colleges provided evidence of teamwork, 
empowerment and measurement in 
this area, with only two colleges having 
high levels of implementation. Strong 
and significant association exists 
between all aspects of leadership, 
as indicated by Table 5. In general, 
a college which displays strengths in 
some aspects of leadership displays 
strengths in other aspects too. 



Market acuity 

□ listening to customers, 
understanding what drives 
value for them, focusing on 
the productivity aspects of 
value as well as the quality 

□ clear competitive positioning 
in the market place 

m listening to staff 
and uses teams 

□ benchmarking against 
other organisations to 
understand better how 
they serve customers 



Table S Association between measures of leadership 





Quality 

leadership 


Supplier 

relationships 


Value 

orientation 


Market 

acuity 


Relationship marketing 


0.669 ♦♦♦ 


0.501 ♦♦♦ 


0.566 ♦♦♦ 


0.607 ♦♦♦ 


Quality leadership 




0.597 ♦♦♦ 


0.522 ♦♦♦ 


0.775^^^ 


Supplier relationships 






0.476 ♦♦ 


0.608 ♦♦♦ 



Value orientation 0.656 ♦ ♦ ♦ 



♦ ♦♦ represents significance at the 0.1% level, ♦♦at the 1% level and ♦ at the 5% level. 
All other correlations are statistically insignificant. 



Service processes 

Service processes is the lowest scoring of the five scales. 
‘Process thinking’ is a key theme in the approaches adopted 
by many of today’s leading service providers, but it seems that 
few colleges have so far adopted a strong process focus in their 
management and improvement activities. The opportunities 
for improvement identified in this area are therefore worthy 
of particular attention. 



Service representations 

□ the tangible elements 
of services such as 
the appearance of the 
buildings and materials 

□ high accessibility 
of those services 



Many colleges have good levels of service representations, 
but a third do not, reflecting low levels of service accessibility 
(physical access and/or hours of operation which are more suited to 
the college and its staff than to the customers) or the standard 
of physical facilities, resources and materials (see Table 6). 
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Table S Distribution of measures for service processes 



Overall score 


Mgmt of key 
processes % 


Kaizen % 


Moments 
of truth % 


New service 
dvipmnt % 


Service 
rprsntns % 


e-busIness 

% 


<50.0 


13 


33 


40 


27 


10 


25 


50.0-59.9 


38 


19 


35 


15 


23 


15 


60.0-69.9 


21 


31 


21 


46 


42 


40 


70.0-79.9 


15 


10 


4 


8 


15 


10 


80.0-89.9 


15 


4 


0 


4 


6 


10 


90.0-100.0 


0 


2 


0 


0 


4 


0 


Total 


100 


100 


100 


100 


100 


100 


Mean 


58.1 


56.8 


50.1 


58.5 


59.6 


58.5 



Poor levels of implementation of new service development practices 
are displayed by 42% of colleges. They do little to involve customers 
in the design or development of new courses and associated 
processes can be ad hoc. 

Management of key processes has similar levels of poor 
implementation, with weaker colleges paying little attention to 
business processes across departments and functions, and using 
information technology (IT) just for standard business applications. 
They could learn from those who focus on defining, mapping 
and redesigning key processes and investing in IT as a means 
of redesigning and substantially improving services, e-business 
systems have been implemented by 60% of colleges, allowing 
customers to search for information through the internet. 

Some have integration across functions and use IT for improved 
communication, but only 10% have very high levels of 
implementation. 

The commitment to and processes of continuous improvement 
are known as kaizen, the driving force for excellence in many 
successful modern organisations. Only 6% of the colleges 
have very good levels of kaizen implementation, while 52% 
have poor levels. 
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These colleges provide limited or no staff education and training 
with a specific ‘quality’ focus, and have not embedded continuous 
improvement within their organisation. Kaizen is one of the poorer 
areas of practice implementation identified in the study, along 
with moments of truth (identifying, mapping and managing key 
interactions between the college and all of its customers) for which 
three-quarters of colleges score poorly. These colleges have not 
fully identified critical points in service delivery processes, or put 
strategies in place for service recovery. They are not proactive in 
nurturing staff loyalty or in providing service guarantees. Only one 
college in six shows good attainment in each of these practices. 

Statistically significant associations (though only moderate in their 
strength) exist between nearly all of the pairs of measures in the 
area of service processes, with the association between kaizen and 
moments of truth being particularly strong (see Table 7). Colleges 
that are strong in some areas of service processes are generally 
strong in others, and the converse is also true. 



Table 7 Association between measures of service processes 





New service 
development 


Mgmt of key 
processes 


e-business 


Kaizen 


Moments 
of truth 


Service representations 


0.439 ♦♦ 


0.386 ♦♦ 


0.373 ♦♦ 


0.419 ♦♦ 


0.510 ♦♦♦ 


New service development 




0.411 ♦♦ 


0.387 ♦♦ 


0.615 ♦♦♦ 


0.619 ♦♦♦ 


Management of key processes 






0.593^^^ 


0.519 ♦♦♦ 


0.543 ♦♦♦ 


e-business 








0.425 ♦♦ 


0.383 ♦♦ 



Kaizen 0.693 ♦♦♦ 



♦ ♦♦ represents significance at the 0.1% level, ♦♦ at the 1% level and ♦ at the 5% level. 
All other correlations are statistically insignificant. 



People 

Many colleges deploy good practices with regard to empowerment 
and being innovative organisations (see Table 8). However, practices 
relating to the cycle of virtue are relatively disappointing, with 44% of 
colleges having poor levels of practice implementation. This cycle 
describes a set of three mutually reinforcing activities: job training 
and education, staff involvement in improvements 
and recognition and reward of staff (see Figure 7). 
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Encouragingly, almost 70% of colleges have good levels of practice 
implementation for each attribute relating to empowerment 



FDgw©6 Cycle of virtue 




It is notable that those colleges that could do more to empower 
their staff tend not to identity and respond to service failure, 
and have yet to develop a culture of staff flexibility to overcome 
the disadvantages of ‘one person, one job’. 



TaW© @ Distribution of measures for people 



Overall score 


Empowerment % 


Cycle of virtue % 


Innovative organisation % 


<50.0 


8 


19 


4 


50.0-59.9 


15 


25 


10 


60.0-69.9 


44 


38 


56 


70.0-79.9 


21 


13 


17 


80.0-89.9 


13 


6 


13 


90.0-100.0 


0 


0 


0 


Total 


100 


100 


100 


Mean 


64.3 


58.9 


65.3 



Colleges with weaker innovative practices do not yet display 
characteristics of a learning organisation. 
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strong and significant levels of association exist between 
the three indices, as shown in Table 9. 

Table 9 Association between measures of people 





Innovative organisation 


Cycle of virtue 


Empowerment 


0.516^^^ 


0.691 ♦♦♦ 


Innovative organisation 




0.734 ♦♦♦ 



♦ ♦♦ represents significance at the 0.1% level, ♦♦ at the 1% level and ♦ at the 5% level. 
All other correlations are statistically insignificant. 
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What do college staff tell us about their colleges? 

Of the first 48 colleges to complete Learning PROBE, 20 also 
participated in a recent staff survey administered by LSDA 
(Owen and Davies 2003). Views were expressed by 2644 
staff of these 20 colleges. 

Many staff think highly of a number of aspects of the college 
and its work, which they also regard as important, including: 

□ understanding their own role and contribution 

□ the support they receive from their own manager 

□ staff working well together in teams 

Ea embedding equal opportunities in college culture. 

However, few aspects included in the survey attracted positive 
responses from a majority of respondents. The following 
aspects exhibit particularly large gaps between staff's 
expectations and their perceptions of the current situation 
in their colleges: 

u staff feeling valued, having their views sought and seeing 
opportunities for progression 

fi involvement in planning and targets, shared goals between academic 
and support staff 

a Job security, ability to take risks without fear of failure 

□ effectiveness of management decision-making, 
communication, provision of necessary information 
and resources. 

Many staff (55%) believe that their colleges are good at 
encouraging feedback from customers: but only 28% believe 
that they act effectively upon that feedback, and 34% that 
complaints are dealt with effectively. 

Colleges classified through Learning PROBE as ‘performance 
leads practice’ and ‘weak/laggards’ have proved likely to have 
the greatest gaps between staff's expectations and their 
assessment of the current reality in their college. This is 
particularly the case in terms of organisational issues such 
as senior management style, communication, college 
infrastructure and use of customer feedback. 
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BEST COPY AVAILABLE 



Performance management 



Balanced scorecard 

Using measures that 
cover both drivers 
(operational performance, 
organisational learning 
and growth) and results 
(financial performance 
and customer/market 
performance) 



The majority of the colleges have implemented good levels of 
practice relating to performance management (see Table 10). 
However, with respect to balanced scorecard implementation, 

36% limit their measurement to a rather narrow, traditional focus 
on things they have to measure rather than a proactive quest for 
knowledge that will help them to improve. Some place little emphasis 
on the implementation of formal quality measurement systems 
outside those required by the funding body or procedures for making 
positive use of customer complaint data to drive improvements. 
These weaker colleges may have systems for reporting only the 
statutory performance requirements and have limited measurement 
of customer satisfaction. There are some good examples of 
balanced scorecard implementation within the sample, with one 
college in eight showing strong levels of implementation across the 
board. 



Over 40% of colleges have poor practices for implementing service 
standards, tending to afford inadequate support for staff, having few 
actual service standards in place or implementing standards that are 
easily achievable. In these colleges, the understanding 
of service quality remains unclear. 

Strong and significant association exists between the two practice 
areas, suggesting that those colleges which have good management 
systems in place are more likely to have a larger number of 
challenging, well-understood service standards. 



Tab)le 10 Distribution of measures for performance management 



Overall score 


Balanced scorecard % 


Service standards % 


<50.0 


13 


17 


50.0-59.9 


23 


25 


60.0-69.9 


46 


42 


70.0-79.9 


6 


13 


80.0-89.9 


10 


4 


90.0-100.0 


2 


0 


Total 


100 


100 


Mean 


62.6 


59.7 
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Results 



Overall, the colleges' operational and business results are better 
than the corresponding levels of practice, suggesting potential 
vulnerability among a substantial number of the colleges, whose 
current performance levels, however good, are not supported 
by comparable systems and practices and may perhaps be 
unsustainable. 

The best results are for business performance, with only 16% of 
colleges scoring poorly (see Table 11). In contrast, 84% perform well 
in terms of margins, returns and positive cash flows and, overall, 
almost a quarter of the colleges have very strong 
business performance. 



Table 10 Distribution of measures for results 



Overall score 


Customer 
growth % 


Service 
quality % 


Organisational 
productivity % 


Results for 
stakeholders % 


Business 
performance % 


<50.0 


4 


6 


11 


15 


10 


50.0-59.9 


31 


21 


11 


31 


6 


60.0-69.9 


35 


44 


61 


29 


40 


70.0-79.9 


19 


25 


11 


13 


21 


80.0-89.9 


10 


4 


7 


10 


21 


90.0-100.0 


0 


0 


0 


2 


2 


Total 


100 


100 


100 


100 


100 


Mean 


62.9 


64.3 


62.6 


57.9 


67.2 



The next best area of performance is organisational productivity. 
Only 22% of the colleges exhibit poor performance, while 79% 
have displayed static/increasing market share and improving 
levels of productivity. 
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Similar performance has been attained in terms of service quality, 
although 27% of colleges score less than 60. These low-scoring 
colleges need to address staff morale, the clarity of their service 
role, how they meet customer needs or the levels of service 
differentiation. Weaker colleges also have a tendency to let down 
their customers, have quality levels which are poor relative to 
the sector, and employ staff who are seen to respond slowly or 
have variable levels of courtesy. They endure customer complaints, 
have decreasing levels of customer satisfaction or have customers 
who perceive lower levels of value. Encouragingly, no colleges 
score poorly in all areas, with one in four having good levels of 
performance, although only 4% attained an index of 80 or more. 

Poor levels of customer growth are shown by 35% of colleges. 

They may have no recent innovations in service, slower rates 
of development, high rates of learner drop-out and loss of local 
partnerships or a declining customer base. Only one college 
has all of these shortcomings, with almost 60% displaying good 
performance and one in ten attaining high levels of growth. 

The poorest levels of performance relate to results for 
stakeholders, reflecting the fact that almost half of the colleges have 
either a high staff turnover or an attention to social and 
environmental issues that is restricted to compliance. 

In general, there are high levels of inter-association between the 
various aspects of performance - if a college achieves good results 
in one area, it is likely to display good results across a range of areas 
(see Table 12). The association between service quality 
and customer growth is particularly strong. 



TaW© 12 Association between measures of results 





Organisational 

productivity 


Service 

quality 


Customer 

growth 


Results for 
stakeholders 


Business performance 


0.408 ♦♦ 


0.155 


0.276 


0.140 


Organisational productivity 




0.489 ♦♦ 


0.420 ♦♦ 


0.230 


Service quality 






0.769 ♦♦♦ 


0.408 ♦♦ 



Customer growth 0.405 ♦♦ 



♦ ♦♦ represents significance at the 0.1% levei, ♦♦ at the 1% level and ♦ at the 5% level. 
All other correlations are statistically insignificant. 
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Benchmarking of colleges has been ongoing since 2000. During 
2002, with support from Tyne and Wear LSC, the programme was 
extended to include WBL providers. The pilot involved deployment 
of Learning PROBE in this new setting, and also the use of a related 
tool, the Micro Business Review^ (MBR), with a number of smaller 
WBL providers. It established that these tools could work well for 
WBL providers, and identified some refinements that would ensure 
their effectiveness and acceptability across the newly enlarged 



learning and skills sector. 

The early months of 2003 
are seeing a new wave of 
benchmarking through Learning 
PROBE and MBR, involving large 
numbers of both WBL providers 
and colleges. This programme 
will generate new findings, 
which will be published in due 
course. Early indications 
are that there will be some 
common ground between 
the patterns of practice and 
performance identified among 
WBL providers and those 
already found among colleges. 



The bemhmsm’kSmig session 
pmvedi to be a noh teBm-bmiUmg 
exeirolse and has added to the 
oontlnulng development 
of the team membeirs. We 
ooveired areas that we have 
not discussed together previously 
and shared information and 
Ideas that will make a difference 
both to our learners and 
the organisation overalL 
WBL provider, 2003 



^ The Micro Business Review/ tool w/as developed by New/castle Business School and 
Business Link County Durham, w/orking w/ith a number of ‘micro businesses', employing 
20 people orfew/er, to develop a benchmarking tool w/hich incorporates the strengths of 
the PROBE approach and is tailored to the needs and language of this size of business. 
It draw/s on methodologies previously developed by NBS and West London TEC for 
small to medium-sized enterprises. 
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Work-based learning providers 

Emerging strengths include: 
a business marragement 
■ employee empowerment and involvement 
a innovative and responsive staff. 

There is room for improvement in several practice 
and results areas: 

a process management, quality-focused leadership, 
performance management 

a service quality, productivity, societal results. 



We Itientifled closely with the main ateas for improvement 
highlighted by the benchmarking survey. They have 
been accounted for in our business plan. This practice 
is what all organisations should be engaging in to 
improve performance and sustain successful results, 

WBL provider, 2003 
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Challenges facing the 
learning and skills sector 




It is clear that learning providers deploy some strong practices 
and achieve much in terms of their organisational performance, 
and that there are many opportunities for one provider to learn 
from others about specific things that they do better, and with 
better results. However, the clearest lessons to emerge so far 
from the Learning PROBE programme are that: 

H on the whole, participating colleges lag behind other UK service 
providers in terms of their practices and performance; there 
is much that the sector could learn by studying the practices 
of strong performers in other sectors 

B the current deployment of Learning PROBE with large numbers of 
WBL providers is likely to identify similar patterns to those displayed 
by the colleges 

m one of the colleges’ weakest practice areas is continuous 

improvement; if the sector is to close the practice and performance 
gap on other service providers, it must first catch up with modern 
practices in the field of quality, improvement and the management of 
change. 

Colleges’ strongest practices relate to their people, with 
empowerment and innovativeness featuring strongly. While this is 
a promising sign, it is accompanied by some warning signs that also 
emerge from the data. First, there is the high proportion of colleges 
whose performance outstrips their practices. Second, there is the 
finding that service processes are colleges’ weakest area of practice. 
Taken together, these findings raise the possibility that relatively 
strong performance is in some cases being ‘bought’ through 
the efforts of staff willing to expend extra efforts to overcome 
the shortcomings of poor systems, practices and/or resources. 
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If this is the reality of life within some learning providers, then the 
sustainability of this situation needs to be questioned, along with 
its cost-effectiyeness in both the shortterm and the longterm. 

Another contributory factor to the high number of colleges whose 
performance leads their practice could be inadequacies in some 
aspects of performance measurement and management. Learning 
providers need to ensure that they are managing by fact, not by 
anecdote or ‘hunch’, in terms of key aspects of the performance 
of their processes and services. 

Although people practices are generally quite strong, some aspects 
of leadership could be stronger, and the virtuous cycle of developing, 
involving and recognising staff lags behind other people practices. 
These current shortcomings will need to be addressed as a matter 
of priority if major changes are to be achieved within the sector, 
and if continuous improvement is to be embedded as a driver 
for future success. 
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Appejito Taking part in Learning PROBE 



Getting approval 

If you would like to take part in Learning PROBE it is important 
that the senior management at your organisation is committed to 
using the tool. Give them a copy of this publication and explain the 
benefits of the process and the staff involvement that it requires. 
Once you have obtained approval contact the LSDA to discuss the 
next steps. You will be sent some information to help you to get 
started on the project. 



The team 

The make-up of the Learning PROBE team is important to 
the success of the project. The team should be multi-level and 
multi-functional, both teaching and administrative/support staff, 
comprising approximately eight to ten people. A coordinator should 
be appointed, who need not necessarily be at senior management 
level. There should be one senior manager or director within 
the team. 

Choosing the team is critical. Individuals should be chosen for 
their knowledge of the learning provider and ability to take part in 
a workshop such as this. The results must be owned by the team, 
and must reflect the position of the learning provider at that time, 
based on a consensus of opinion from the team. Once the team has 
been decided, contact the LSDA to arrange a date for the facilitator 
to visit (this can be done sooner if necessary). 

The questionnaires will now be sent out to you. 
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The questionnaire 

The team’s job is now to complete the questionnaire. A copy should 
be given, with a covering memo/verbal explanation, to each team 
member. It is suggested that each member reads and attempts 
to answer all questions, but if there are a few specific questions 
that some individuals simply cannot answer this is not a problem. 

As appropriate, team members should collect any data relevant to 
their area that may be needed for completion of the questionnaire. 

If there is no finance-based person on the team the finance team 
may have to be contacted to help with some of the questions. 

The coordinator should also complete the coordinator questionnaire 
and should submit this to the facilitator in advance of the facilitation 
day if possible, or at the latest at the beginning of the facilitation day. 
The coordinator may consult colleagues about the answers 
to these sections, and/or may wish to involve the whole team 
in discussing some of them. 

The team should convene before the facilitation day to go through the 
main questionnaire. This will take approximately 2-4 hours, 
so it is wise to leave half a day free. This is to check that there is 
a full and common understanding of the questions, and to identify 
questions for which there is already a broad consensus, and those 
for which there is a wide range of scoring. The reasons behind 
scoring differences should be discussed, but it is not necessary to 
arrive at consensus scores for all of the questions at this meeting. 

Once the questionnaire and initial scores have been fully discussed, 
nothing should be done until the facilitation day, which will take place 
at the learning provider’s premises. 
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The facilitation day 

The facilitation session takes place to ensure that the quality of data 
being entered into the database is maintained. It also ensures that 
the team has complete understanding of the questions as applied to 
their learning provider. The facilitators are fully trained with current 
knowledge of best practices and benchmarking processes. 
Facilitators may work alone, but regularly ‘double up’ 
to ensure that a consistent approach continues to be applied. 

□ Initially the facilitator should have a short introduction to the learning 
provider (explanation of services and functions; introduction to a 
range of personnel; tour of facilities if appropriate). Following this, 
he/she can meet the team and proceed to facilitate a discussion of 
the questionnaire, leading to verification of consensus scores. It is 
helpful for the facilitator to have a record of the range of scores that 
were awarded to each question by each team member (preferably in 
advance of the day), so that he/she can establish which questions 
may need greater discussion in order to reach the consensus required. 

□ Once the session is completed, which typically takes around 4-5 hours, 
the facilitator will need approximately 1 hour to prepare the analysis. 

□ At the final feedback session the facilitator will report to the team 
the results of their self-assessment. 



The report 

At this point, the ‘on-site’ process is complete. The facilitator will 
take the results away for further analysis. A report will be produced 
and will be forwarded along with the relevant charts within 2 weeks 
of the facilitation day. 



Next steps 

It is important that the report is not the end of the process. You should 
arrange to meet, if possible as a whole team, to discuss the report and 
to decide how best to present the results and take the project forward. 
At this point the original management commitment will pay dividends. 

Organise an action planning session that involves senior managers 
to identify priorities that have been recognised and develop solutions 
and improvements. 
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Learning PROBE steps 




Additional information can be obtained by calling Jane Owen 
on 020 7297 9083 or e-mail jowen@LSDA.org.uk 
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Useful links 



The Learning and Skills Development Agency offers support 
to learning providers through its RQA (Raising Quality and 
Achievement) Programme, including helping providers to join 
the Learning PROBE benchmarking programme and to make 
use of related tools and approaches. 

Cofluftact Jane Owen 

020 7297 9083 

jowen@LSDA.org.uk 

www.LSDA.org.uk 



The Centre for Business Excellence at Newcastle Business School, 
Northumbria University, researches the effectiveness of approaches 
to continuous improvement and the diffusion of good practice and 
innovation, working with partners and clients to identify and apply the 
best available ideas and test them in real-world situations. 

CoBDltaet David Yarrow 

0191 227 3922 

david.yarrow@northumbria.ac.uk 

www.northumbria.ac.uk 

The CBI supports the deployment of Learning PROBE as an 
extension to its Best Practice activities, which include making 
the PROBE suite of benchmarking tools available to its members. CBI 
draws upon the research gathered through PROBE as an input on its 
lobbying and good practice exchange and dissemination activities. 

CoBDltacft Mindy Wilson 

020 7395 8186 

Mindy.Wilson@cbi.org.uk 

www.cbi.org.uk 
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raising quality 
and achievement 
programme 
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If learning providers can become learning organisations, 
their services will be delivered more and more effectively. 



With this in mind, the Learning and Skills Development Agency 
selected PROBE, a tool that has assisted 3000 organisations 
across the world to review their performance and practices, and 
worked with its creators to develop Learning PROBE, a version 
for the learning and skills sector. 



This publication explains how PROBE was developed and adapted 
for the sector, resulting in the use of Learning PROBE and its 
small business version by almost 100 colleges and work-based 
learning providers by April 2003. It also reflects the lessons learnt 
by the first 48 learning providers to complete the Learning PROBE 
benchmarking process. 
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